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TEXT REFLOW IN A STRUCTURED DOCUMENT 

BACKGROUND OF THE INVENTION 
The present invention relates generally to a system and method for generating output 
5 for computer systems, and more particularly to a system and method for modifying the 
presentation of structured documents. 

The rapid expansion of the World Wide Web - where dynamic, compelling images 
are crucial - has driven the demand for a document format that preserves all of the fonts, 
formatting, colors, and graphics of any source document, regardless of the application and 
1 0 platform used to create it. 

One such format is referred to as the portable document format (PDF). PDF is a file 
format developed by Adobe Systems, Incorporated. PDF captures formatting information 
from a variety of desktop publishing applications, making it possible to send formatted 
documents and have them appear on the recipient's monitor or printer as they were intended. 
15 A source document can be authored in a page description language (PDL). PDL is a 

language for describing the layout and contents of a printed page. One well-known PDL is 
PostScript™ by Adobe Systems, Inc. PostScript describes a page in terms of page objects 
including textual objects and graphical objects such as lines, arcs, and circles. Moreover, 
PDF is a PDL. 

20 According to conventional methods, a PDL document is rendered by first defining a 

bounding box, such as a page, and then placing the textual and graphical objects defined for 
the page into the bounding box according to the definitions in the PDL document. Normally 
the size of a PDL page is chosen so that it is clearly legible on a standard full-sized display. 
For example, an 8-inch page width may be chosen for display on a 15-inch computer 

25 monitor. 

However, a single page size may not be ideal for devices having non-standard display 
sizes, such as a hand-held personal digital assistant (PDA). To view a PDL page having an 
8-inch page width on a 3-inch-wide display, the user has two alternatives. The user can 
change the zoom factor of the display to show the entire page on the 3 -inch display. 
30 However, this approach will generally render the page too small to be legible. Alternatively, 
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the user can simply show a portion of the page at full magnification, and scroll horizontally 
and vertically to view the rest of the page. This approach is inconvenient and time- 
consuming. 

Another approach is for the author of the PDL page to generate a different PDL page 
5 for each display size using the application that created the PDL page. This approach is 

wasteful because multiple copies of each PDL page must be maintained. In addition, the user 
must select the copy that is appropriate for the display on which the PDL page will be 
viewed. 

SUMMARY OF THE INVENTION 

10 In general, in one aspect, the invention features a method and computer program 

product for reflowing a PDL page without using the original application used to create the 
PDL page. One use of the present invention is to resize a page for viewing on a display of a 
different size than the display to which the document was originally rendered, or to display at 
a different resolution, either because the inherent resolution of the display differs, or because 

15 the reader has demanded a larger, more visible representation. 

In one aspect the method and computer program product include receiving a page 
represented in a page description language, the page including a plurality of page objects; 
and changing a size of the page to a changed size in a first dimension without changing the 
size of the page objects, while maintaining spatial relationships between the page objects in a 

20 second dimension. 

Particular implementations can include one or more of the following features. The 
page objects can include textual and graphical elements, and the changing step includes 
maintaining spatial relationships between the textual and graphical elements in the second 
dimension. The changing step can further include creating one or more new pages having the 

25 changed size in the first dimension; and adding the textual and graphical elements to the one 
or more new pages. The adding step can include identifying distances in the second 
dimension between one or more textual elements and one or more graphical elements; adding 
the textual elements to the one or more new pages; and adding the graphical elements to the 
one or more new pages based on the distances in the second dimension and positions of the 

30 textual elements in the one or more new pages. The identifying step can include creating a 
map containing the positions of the textual elements in the page, and augmenting the map 
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with the positions of the textual elements in the one or more new pages to produce a 
relationship for each textual element between the position of the textual element in the page 
and the position of the textual element in the one or more new pages; and the step of adding 
the graphical elements can include adding the graphical elements to the one or more new 
5 pages according to the map. The step of adding the graphical elements according to the map 
can include associating one or more particular textual elements with one or more particular 
graphical elements; determining distances in the second dimension between the particular 
textual elements and the particular graphical elements; and selecting positions in the one or 
more new pages for placement of the particular graphical elements based on the distances 
10 and the positions of the particular textual elements in the one or more new pages as listed in 
the map. The associating step can include associating a particular graphical element with a 
particular textual element that is nearest to the particular graphical element in the first 
dimension. The associating step can include associating a given graphical element with a 
f ; plurality of given textual elements; and the selecting step can include scaling the given 

111 1 5 graphical element when a distance in the first dimension between the plurality of given 
Jl textual elements in the page differs from a corresponding distance between the plurality of 

^ given textual elements in the one or more new pages. The textual elements can be organized 

O as words. 

5 In another aspect the method and computer program product include receiving a page 

tfJ 20 represented in a page description language, the page including a plurality of page objects; 
q and changing a size of the page objects without changing the size of the page, while 

maintaining spatial relationships between the page objects in a dimension of the page. The 
page objects can include textual and graphical elements, and the changing step can include 
maintaining spatial relationships between the textual and graphical elements in the 
25 dimension. The changing step can include creating one or more new pages having the same 
size as the page in a further dimension; scaling the textual and graphical elements, producing 
scaled textual and graphical elements; and adding the scaled textual and graphical elements 
to the one or more new pages. The adding step can include identifying distances in the 
dimension between one or more textual elements and one or more graphical elements; adding 
30 the scaled textual elements to the one or more new pages; and adding the scaled graphical 
elements to the one or more new pages based on the distances in the dimension and the 
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positions of the scaled textual elements in the one or more new pages. The identifying step 
can include creating a map containing the positions of the textual elements in the page, and 
augmenting the map with the positions of the textual elements in the one or more new pages 
to produce a relationship for each textual element between the position of the textual element 

5 in the page and the position of the corresponding scaled textual element in the one or more 
new pages; and the step of adding the graphical elements can include adding the graphical 
elements to the one or more new pages according to the map. The step of adding the scaled 
graphical elements according to the map can include associating one or more particular 
textual elements with one or more particular graphical elements; determining distances in the 

1 0 dimension between the particular textual elements and the particular graphical elements; and 
selecting positions in the one or more new pages for placement of the scaled graphical 
elements corresponding to the particular graphical elements based on the distances and the 
positions of scaled textual elements corresponding to the particular textual elements in the 
one or more new pages as listed in the map. The associating step can include associating a 

1 5 particular graphical element with a particular textual element that is nearest to the particular 
graphical element in the further dimension. The associating step can include associating a 
given graphical element with a plurality of given textual elements; and the selecting step can 
include scaling the given graphical element in the dimension when a distance in the 
dimension between the plurality of given textual elements in the page differs from a 

20 corresponding distance in the one or more new pages between a plurality of scaled textual 
elements corresponding to the plurality of given textual elements. The textual elements can 
be organized as words. 

In another aspect the method and computer program product include receiving a page 
represented in a page description language, the page including a plurality of page objects; 

25 and changing a size of the page to a changed size in a first dimension, and changing a size of 
one or more of the page objects, while maintaining spatial relationships between the page 
objects in a second dimension. The page objects include textual and graphical elements, and 
wherein the changing step can include maintaining spatial relationships between the textual 
and graphical elements in the second dimension. The changing step can include creating one 

30 or more new pages having the same size as the page in a further dimension; scaling the 

textual and graphical elements, producing scaled textual and graphical elements; and adding 
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the scaled textual and graphical elements to the one or more new pages. The adding step can 
include identifying distances in the second dimension between one or more textual elements 
and one or more graphical elements; adding the scaled textual elements to the one or more 
new pages; and adding the scaled graphical elements to the one or more new pages based on 

5 the distances in the second dimension and positions of the textual elements in the one or 
more new pages. The identifying step can include creating a map containing the positions of 
the textual elements in the page, and augmenting the map with the positions of the textual 
elements in the one or more new pages to produce a relationship for each textual element 
between the position of the textual element in the page and the position of the corresponding 

1 0 scaled textual element in the one or more new pages; and the step of adding the graphical 
elements can include adding the graphical elements to the one or more new pages according 
to the map. The step of adding the graphical elements according to the map can include 
associating one or more particular textual elements with one or more particular graphical 
elements; determining distances in the second dimension between the particular textual 

15 elements and the particular graphical elements; and selecting positions in the one or more 
new pages for placement of the scaled graphical elements corresponding to the particular 
graphical elements based on the distances and the positions of scaled textual elements 
corresponding to the particular textual elements in the one or more new pages as listed in the 
map. The associating step can include associating a particular graphical element with a 

20 particular textual element that is nearest to the particular graphical element in the first 

dimension. The associating step can include associating a given graphical element with a 
plurality of given textual elements; and the selecting step can include scaling the given 
graphical element when a distance in the dimension between the plurality of given textual 
elements in the page differs from a corresponding distance in the one or more new pages 

25 between a plurality of scaled textual elements corresponding to the plurality of given textual 
elements. The textual elements can be organized as words. The amount of size change of a 
page object can depend on the type of the page object. 

In general, in one aspect, the invention features a method and computer program 
product for forming illustrations in a page. It includes receiving a page represented in a page 

30 description language (PDL), the page including a plurality of page objects including line art 
elements, each page object associated with a PDL element range including at least one PDL 
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element; repeatedly augmenting each PDL range with a PDL element that is adjacent to the 
PDL range and is not part of another PDL range when the bounding box of the PDL element 
overlaps the bounding box of the PDL object associated with the PDL range; such that the 
PDL elements in each PDL range define an illustration. Particular implementations can 
5 include combining two illustrations when their PDL ranges are adjacent and their bounding 
boxes overlap. 

In another aspect the method and computer program product include receiving a page 
represented in a page description language (PDL), the page including a plurality of page 
objects including line art elements, each page object associated with a PDL element range 

1 0 including at least one PDL element; recursively coalescing line art elements having 

overlapping bounding boxes to form one or more illustrations; adding to each illustration 
each PDL element within the PDL range of the illustration that is not part of the illustration 
when the bounding box of the PDL element overlaps the bounding box of the illustration; 
and repeatedly augmenting the PDL range of each illustration with PDL elements that are 

15 adjacent to the PDL range and are not part of another illustration when the bounding box of 
the PDL elements overlap the bounding box of the illustration; such that the PDL elements in 
each PDL range define an illustration. Particular implementations can include combining 
two illustrations when their PDL ranges are adjacent and their bounding boxes overlap. The 
step of recursively coalescing can include combining two line art elements having 

20 overlapping bounding boxes, thereby forming an illustration; and creating a new bounding 
box containing the illustration. Particular implementations can include combining a line art 
element with the illustration when the bounding boxes of the line art element and the 
illustration overlap. 

Advantages that can be seen in implementations of the invention include one or more 
25 of the following. PDL pages can be resized. A PDL page produced according to the 
invention can be legibly displayed on any size display. 

The details of one or more embodiments of the invention are set forth in the 
accompanying drawings and the description below. Other features and advantages of the 
invention will become apparent from the description, the drawings, and the claims. 



30 



BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 is a block diagram of a computing system for displaying a document. 

6 



Attorney's Docket No. 07844-366001 



FIG. 2 depicts a conventional PDL page. 

FIG. 3 depicts a PDL page resulting from the operation of one implementation of the 
present invention. 

FIG. 4 depicts a PDL page resulting from the operation of another implementation of 
5 the present invention. 

FIG. 5 depicts a PDL page resulting from the operation of yet another implementation 
of the present invention. 

FIG. 6 is a flowchart depicting a process for reflowing a PDL page. 

FIGS. 7-10 show the results of the process of FIG. 6 when applied to a PDL page 
1 0 containing eleven PDL elements. 

FIG. 1 1 depicts a process for forming illustrations. 

FIG. 12 is a flowchart depicting an alternative process for forming illustrations. 
Like reference numbers and designations in the various drawings indicate like 
elements. 

1 5 DETAILED DESCRIPTION 

The present invention can be implemented in a raster image processor (RIP). This 
implementation is described with reference to FIG. 1. 

FIG. 1 is a block diagram of a computing system 100 for displaying a document. In 
system 100, an application 102, such as a word processor, generates a request to display a 

20 page in response to a user selection. The request is sent to an operating system 1 04, such as 
Windows™, using an operating system call. The operating system 104 in turn directs the 
request to a page description language (PDL) driver 106. The driver 106 translates the 
operating system calls and generates a data stream that is sent to RIP 120. RIP 120 operates 
to generate an encoded raster file 122. The encoded raster file is sent to display device 130 

25 for display. 

The implementation described below changes only the page width, and not the size of 
the page objects. In another implementation, only the size of the page objects is changed. In 
yet another implementation, both the page width and the size of the page objects are changed. 
In variations of these implementations, it is the page length that is changed, rather than the 
30 page width. These variations are especially useful for languages that are written from top to 
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bottom, rather than from left to right. The textual elements can be organized as characters, 
words and the like. 

FIG. 2 shows a conventional PDL page 202. The page includes textual elements 206 
and graphical elements 208 and 210. The page has a page width 220. Each graphical 

5 element in a page is associated with one or more textual elements in the page. Graphical 
element 208 spans several lines of text. Therefore, it is associated with two textual elements 
212 ("I") and 214 ("desperate"). For convenience, textual elements 212 and 214, referred to 
as "anchors," are shown in bold type. Graphical element 210 spans only a single line. 
Therefore, it is associated with a single anchor 216 ("My"). 

10 FIG. 3 shows a PDL page 302 after application of the reflow process of the present 

invention. The width of the original PDL page 202 is changed while the font size of a textual 
element 206 remains constant. Referring to FIGS. 2 and 3, the page width 320 of new page 
302 is chosen to be narrower than the page width 220 of original page 202. Accordingly, the 
length of the page increases. When referring to a PDL page, the horizontal dimension is 

15 referred to as its X-axis and the vertical dimension is referred to as the Y-axis. Thus, it is the 
distance in the Y-axis that increases. In this implementation, the graphical elements are 
moved and stretched to follow their anchors. 

The Y-axis position of anchor 216 has also increased. Therefore, the Y-axis position 
of associated graphical element 210 is also increased so that anchor 216 and graphical 

20 element 210 are aligned on the Y-axis. In addition, the Y-axis distance between anchors 212 
and 214 has increased. Therefore, corresponding graphical element 208 is stretched to span 
the distance between anchors 212 and 214. 

FIG. 4 shows a PDL page 402 resulting from the operation of another implementation 
of the present invention. In this implementation, the width of the PDL page remains constant 

25 while the font size of its textual elements is increased. Referring to FIGS. 2 and 4, the page 
width 420 of page 402 is the same as the page width 220 of page 202 (FIG. 2). However, the 
font size of the textual elements 206 has been increased from 12 points to 14 points. 
Therefore, the length of the page 402 is greater than the length of page 202. 

The Y-axis position of anchor 216 has also increased. Therefore, the Y-axis position 

30 of associated graphical element 210 is increased so that anchor 216 and graphical element 
210 are aligned on the Y-axis. In addition, the Y-axis distance between anchors 212 and 214 
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has increased. Therefore, corresponding graphical element 208 is stretched to span the 
distance between anchors 212 and 214. 

FIG. 5 shows a PDL page 502 resulting from the operation of another implementation 
of the present invention. According to this implementation, the width of the PDL page and 
5 the font size of its textual elements are increased. Referring to FIGS. 2 and 6/5, the page 
width 520 of new page 502 is chosen to be narrower than the page width 220 of original page 
202. In addition, the font size of the textual elements 206 has been increased from 12 points 
to 14 points. Therefore, the length of the page 502 is greater than the length of page 202. 

The Y-axis position of anchor 216 has also increased. Therefore, the Y-axis position 
10 of associated graphical element 210 is increased so that anchor 216 and graphical element 
210 are aligned on the Y-axis. In addition, the Y-axis distance between anchors 212 and 214 
has increased. Therefore, corresponding graphical element 208 is stretched to span the 
distance between anchors 212 and 214. 

FIG. 6 is a flowchart depicting a process for reflowing a PDL page. According to this 
15 process, only the width of the PDL page is changed. The size of the textual elements within 
the PDL page remains constant. In another implementation, the size of the textual elements 
is increased while the page width is unchanged, as shown in FIG. 4. In another 
implementation, both the page width and the font size of the textual elements are changed, as 
shown in FIG. 5. 

20 A PDL page is received in step 602. The PDL page includes page objects and 

including textual and graphical elements. In step 604, the process creates a new PDL page 
having a page width that differs from that of the received original PDL page. Note that all of 
the implementations maintain the spatial relationship between the textual and graphical 
elements. 

25 In one implementation, two or more new PDL pages are created. The textual 

elements are added to the new PDL page in step 606. Next sub-process 620 is repeated for 
each graphical element. 

The graphical element is associated with one or more textual elements in step 608. 
Then the process determines a Y-axis difference between the position of the graphical 

30 element and a position of each associated textual element, as shown in step 610. This step is 
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useful when a graphical element and its associated textual element do not have the same Y- 
axis position. 

In one implementation, the Y-axis positions of each textual element in the original 
PDL page and the new PDL page are recorded in a "Y-map." An exemplary Y-map is shown 

5 in Table 1. Table 1 is a Y-map between page 202 of FIG. 2 and page 302 of FIG. 3. Table 1 
presents the Y position in the old page (page 202) and the Y position in the new page (page 
302) for the anchor textual elements. In another implementation, the Y-map records 
positions for each textual element. Referring to Table 1, it is seen that the textual element "I" 
(anchor 212) is at a Y-axis position of 0.25 in both the old and new pages. However, the 

10 word "desperate" (anchor 214) has moved by half an inch. Similarly, the word "My" (anchor 
216) has moved by % of an inch. 



Textual Element 


Y Position in Old Page 


Y Position in New Page 


I 


0.25 


0.25 


desperate 


1.75 


2.25 


My 


2.00 


2.75 



Table 1 



A position for the graphical element in the new page is then determined based on the 
Y-axis position of the associated textual element(s) in step 612. For example, the position of 

15 graphical element 210 in page 302 is determined based on the Y-axis position of its 

associated textual elements (anchor 216) in page 302. Referring to the Y-map of Table 1, it 
is seen that the Y-axis position of anchor 216 is 2.75 inches. Therefore, assuming that the Y- 
axis difference between graphical element 210 and anchor 216 is zero, the position for 
graphical element 210 in page 302 is determined to be 2.75 inches along the Y-axis. If 

20 necessary, the graphical element is scaled (that is, stretched) in step 614. For example, 

referring to FIG. 3, graphical element 208 is stretched because its associated anchors 212 and 
214 are further apart than in the original page. Finally, the graphical element is placed at the 
position determined in step 612, as shown in step 616. 

The present invention is also useful in documents having multiple columns. Each 

25 column is simply treated according to the process of FIG. 6 within its bounding box in the 
same way the PDL page 202 is treated within page 302. Implementations of the present 
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invention include three variations to handle different multi-column cases. For example, 
consider a two-column case with a vertical line-art element between the two columns. Each 
column has a Y-map, so there are two Y-maps. The vertical line-art element is associated 
with textual elements in both columns. When reflowing the line-art element, it must be 

5 decided which Y-map to use. In one implementation, the Y-map having the maximum Y 
value for the line-art element is used. Other implementations can be used, as would be 
apparent to one skilled in the relevant arts. 

In a multiple column case, where a single text-line or line-art element intersects the 
X-axis extent of a set of columns, that intersection is considered to be a "fault line." This 

10 fault line is used as a break, so that subsequent text (that is, text that has a greater Y-axis 
position than the fault line) is considered to be a new column. 

An implementation of the present invention handles hyphenation and ligatures 
according to the following method. The prdcess hyphenates a word at a line-end in the new 
page if the original word had a soft hyphen at that point. Similarly, such a word can be 

15 unhyphenated when it is no longer at a line-end in the new page. In implementations in 
which the font size of the text elements is changed, the positioning of the text lines is 
addressed. In one implementation, all vertical white space is retained. That is, the inter- 
paragraph vertical gap is retained, and within the paragraph, the inter-line vertical gap is 
retained. 

20 In the implementation discussed above, the graphical elements can be simple line art 

elements, images, or combinations of line art elements, images and text, such as captions. 
PDF documents often contain complex illustrations including multiple graphical elements, 
such as strokes and fills, images, and even textual elements, such as captions. Humans are 
very good at identifying which components belong to a particular illustration. However, in 

25 order to successfully reflow a document containing a complex illustration, it is desirable to 
perform this process automatically. After an illustration is identified, it can be reflowed into 
the new page according to the process described above. 

FIGS. 8-10 graphically depict combining page objects to form a single illustration. 
FIG. 7 represents a PDL page containing eleven PDL elements. PDL elements El, E2, E3, 

30 E4, E5 and E6 are line art elements forming a "stick man" standing on the ground. PDL 

elements E7 and E9 are line art elements representing balloon tethers. PDL elements E8 and 



11 



Attorney's Docket No. 07844-366001 



E10 are images representing balloons. PDL element El 1 is a text element that is the caption 
for the drawing. 

The process begins by coalescing line art elements to form illustrations. Each line art 
element is assigned a "bounding box." In the described implementation, each bounding box 
5 is rectilinear having sides parallel to the edges of the PDL page. Referring to FIG. 8, PDL 
element E7 is enclosed by bounding box 802, and PDL element E9 is enclosed by bounding 
box 804. 

According to the process, when line art elements have overlapping bounding boxes, 
they are combined to form a single illustration. This illustration, 12, contains the elements E7 
1 0 and E9 and can be represented by 

12 = E7, E9 (1) 

The drawing of FIG. 7 is represented by a sequential collection of PDL elements 
given by equation 1 . Assume that PDL elements forming the stickman standing on the 
ground have already been collected to form an illustration II given by 

II =E1,E2,E3,E4,E5,E6 (2) 

The sequence of PDL elements, from first to last, associated with an illustration is 
referred to as its "range." The PDL representation of the drawing of FIG. 7 is given below 
with the range of illustrations II and 12 underlined. 

PDL - El, E2, E3, E4, E5, E6, E7, E8, E9, E10, El 1 (3) 

20 Note that, although element E8 is not part of illustration 12, it falls within the range of 

illustration 12. The treatment of such "gaps" in range is described below. 

The process recursively joins line-art elements to form illustrations. Referring to 
FIG. 9, illustration 12 has been enclosed in bounding box 902, and illustration II has been 
enclosed in bounding box 904. Because bounding boxes 902 and 904 overlap, the process 

25 coalesces the contained line-art elements to form an illustration 13 given by equation 6. 

13 - El, E2, E3, E4, E5, E6, E7, E9 (4) 
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The PDL range of illustration 13 then runs from El to E9. The PDL sequence for 
FIG. 9 is given below with the range of illustration 13 underlined. 

PDL = EL E2. E3. E4, E5, E6, E7, E8, E9 , E10, El 1 (5) 

The process next seeks to fill gaps in the range such as that represented by element 
E8. At this point, all of the line-art elements in the PDL page have been coalesced to form 
illustrations. Now the process seeks to combine those illustrations with other PDL elements 
such as images and text. The first step of this process is to fill the gaps in the PDL ranges of 
the line-art illustrations. Referring to FIG. 10, PDL element E8 is an image that lies within 
the range of illustration 13. The process combines a PDL element with an illustration if it lies 
within the PDL range of that illustration and the bounding boxes of the illustration and the 
PDL element overlap. Referring to FIG. 10, illustration 13 has been enclosed in a bounding 
box 1002, and image E8 has been enclosed within a bounding box 1004. Bounding boxes 
1002 and 1004 overlap. Therefore, element E8 is combined with illustration 13. Illustration 
13 now combines all of the elements from El to E9 as shown below. 

13 - El, E2, E3, E4, E5, E6, E7, E8, E9 (6) 

The PDL representation of FIG. 10 is given below, with the range of 13 underlined. 

PDL = EL E2, E3, E4, E5, E6, E7, E8, E9 , E10, El 1 (7) 

The process then seeks to combine PDL elements that are adjacent to the range of an 
illustration. A PDL element will be combined with an illustration when it is adjacent to the 
illustration in the PDL sequence and the bounding boxes of the illustration and the PDL 
element overlap. Referring to equation 9, we see that image E10 is adjacent to the range of 
illustration 13. Referring to FIG. 10, we see the bounding box 1006 of image E10 and the 
bounding box 1002 of illustration 13 overlap. Therefore, image E10 is combined with 
illustration 13. As a result, illustration 13 is given by 

13 = El, E2, E3, E4, E5, E6, E7, E8, E9, E10 (8) 

The PDL representation of the drawing is then given by 
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PDL = El, E2, E3, E4, E5, E6, E7, E8, E9, E10 , El 1 (9) 

Referring to equation 11, it is seen that text element Ell is adjacent to the range of 
illustration 13. Referring to FIG. 10, it is seen that the bounding box 1008 of element Ell 
and bounding box 1002 illustration 13 overlap. Therefore, text element Ell is combined with 
5 illustration 13. Illustration 13 has been given by 

13 =E1, E2 ? E3, E4, E5, E6, E7 5 E8, E9, E10, Ell (10) 

The PDL range of the drawing has been given by 

PDL = El, EZ E3, E4, ES, E6. E7, E8, E9, Eld El 1 (11) 

It is seen that all of the elements within the drawing have now been combined to form 
10 a single illustration. 

FIG. 1 1 depicts a process for forming illustrations according to an implementation of 
the present invention. A PDL page is received at step 1102. The PDL page includes page 
objects including line-art elements. The page objects can also include other sorts of PDL 
elements such as images and text elements. Each page object is associated with a PDL 
15 element range that includes at least one PDL element. It is possible that an illustration 
generated in this way is simply part of a background. Therefore, if a page element 
illustration is the same color as the background, it is discarded in step 1 103. 

The line-art elements within the PDL page are recursively coalesced to form 
illustrations at step 1104. The first recursion includes combining two line-art elements when 
20 their bounding boxes overlap to form an illustration. Following recursions include 

combining the illustration and a line-art element when their bounding boxes overlap to form 
the next illustration. 

As described above, the range of an illustration may include PDL elements that are 
not part of the illustration. These PDL elements represent "gaps" in the PDL range. These 
25 "gaps" are located at step 1 1 05. Each such "gap" is tested to determine whether it should be 
joined with the illustration at step 1 106. A PDL element within the range of an illustration is 
combined with that illustration when the bounding boxes of the element and the illustration 
overlap. 
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The process then tests PDL elements that are adjacent to the ranges of the illustrations 
to determine whether they should be joined with those illustrations in step 1 108. In one 
implementation, only those elements that are not already a part of another illustration are 
tested. A PDL element that is adjacent to the range of an illustration is combined with that 
illustration when the bounding boxes of the element and illustration overlap. This step is also 
performed recursively to "grow" the illustrations as the range of each illustration increases. 

The process then tests the illustrations to determine whether any of them should be 
joined to form a single illustration in step 1110. Illustrations are combined when their PDL 
ranges are adjacent and their bounding boxes overlap. 

Each PDL range that results from this process is a separate illustration. Each 
illustration is treated as a separate graphical element in the reflow process described above. 

FIG. 12 is a flowchart depicting a process for forming illustrations according to 
another implementation of the present invention. A PDL page is received at step 1202. The 
PDL page includes page objects including line-art elements. The page objects can also 
include other sorts of PDL elements such as images and text elements. Each page object is 
associated with a PDL element range that includes at least one PDL element. 

The process then tests PDL elements that are adjacent to the ranges of the illustrations 
to determine whether they should be joined with those illustrations in step 1204. In one 
implementation, only those elements that are not already a part of another illustration are 
tested. A PDL element that is adjacent to the range of an illustration is combined with that 
illustration when the bounding boxes of the element and illustration overlap. This step is also 
performed recursively to "grow" the illustrations as the range of each illustration increases. 

According to one implementation, each illustration is classified for treatment during 
reflow. Illustrations that lie within one line of a paragraph are classified as character 
surrogates. A character surrogate is an illustration that functions as a text character. 
Character surrogates are treated as text elements during reflow. Illustrations that lie within 
the bounding box of a paragraph, and which vertically overlap with two or more initial lines, 
and which are to the left of all the characters in those lines, are classified as "illuminated 
letters." Illuminated letters are reflowed at the upper left of the paragraph, with the text 
elements of the paragraph flowed to the right and below the illustration. 
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Illustrations that lie directly above (or below) a paragraph are reflowed to lie directly 
above (or below) that paragraph. Illustrations that do not fit within the reflow bounding box 
are scaled to fit within that bounding box. 

The invention can be implemented in digital electronic circuitry, or in computer 

5 hardware, firmware, software, or in combinations of them. Apparatus of the invention can be 
implemented in a computer program product tangibly embodied in a machine-readable 
storage device for execution by a programmable processor; and method steps of the invention 
can be performed by a programmable processor executing a program of instructions to 
perform functions of the invention by operating on input data and generating output. The 

10 invention can be implemented advantageously in one or more computer programs that are 
executable on a programmable system including at least one programmable processor 
coupled to receive data and instructions from, and to transmit data and instructions to, a data 
storage system, at least one input device, and at least one output device. Each computer 
program can be implemented in a high-level procedural or object-oriented programming 

15 language, or in assembly or machine language if desired; and in any case, the language can 
be a compiled or interpreted language. Suitable processors include, by way of example, both 
general and special purpose microprocessors. Generally, a processor will receive instructions 
and data from a read-only memory and/or a random access memory. Generally, a computer 
will include one or more mass storage devices for storing data files; such devices include 

20 magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and 
optical disks. Storage devices suitable for tangibly embodying computer program 
instructions and data include all forms of non-volatile memory, including by way of example 
semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; 
magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and 

25 CD-ROM disks. Any of the foregoing can be supplemented by, or incorporated in, ASICs 
(application-specific integrated circuits). 

To provide for interaction with a user, the invention can be implemented on a 
computer system having a display device such as a monitor or LCD screen for displaying 
information to the user and a keyboard and a pointing device such as a mouse or a trackball 

30 by which the user can provide input to the computer system. The computer system can be 
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programmed to provide a graphical user interface through which computer programs interact 
with users. 

The invention has been described in terms of particular embodiments. Other 
embodiments are within the scope of the following claims. For example, the steps of the 
invention can be performed in a different order and still achieve desirable results. 
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WHAT IS CLAIMED IS: 



1 1 . A computer program product, tangibly stored on a computer-readable 

2 medium, for reflowing a page, comprising instructions operable to cause a programmable 

3 processor to: 

4 receive a page represented in a page description language, the page including a 

5 plurality of page objects; and 

6 change a size of the page to a changed size in a first dimension without changing the 

7 size of the page objects, while maintaining spatial relationships between the page objects in a 

8 second dimension. 

1 2. The computer program product of claim 1, wherein the page objects include 

2 textual and graphical elements, and wherein the instructions operable to cause a 

3 programmable processor to change comprise instructions operable to cause a programmable 

4 processor to: 

5 maintain spatial relationships between the textual and graphical elements in the 

6 second dimension. 

1 3 . The computer program product of claim 2, wherein the instructions operable 

2 to cause a programmable processor to change further comprise instructions operable to cause 

3 a programmable processor to : 

4 create one or more new pages having the changed size in the first dimension; and 

5 add the textual and graphical elements to the one or more new pages. 

1 4. The computer program product of claim 3, wherein the instructions operable 

2 to cause a programmable processor to add comprise instructions operable to cause a 

3 programmable processor to : 

4 identify distances in the second dimension between one or more textual elements and 

5 one or more graphical elements; 

6 add the textual elements to the one or more new pages; and 

7 add the graphical elements to the one or more new pages based on the distances in the 

8 second dimension and positions of the textual elements in the one or more new pages. 
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1 5 . The computer program product of claim 4, wherein: 

2 the instructions operable to cause a programmable processor to identify include 

3 instructions operable to cause a programmable processor to 

4 create a map containing the positions of the textual elements in the page, and 

5 augment the map with the positions of the textual elements in the one or more 

6 new pages to produce a relationship for each textual element between the position of 

7 the textual element in the page and the position of the textual element in the one or 

8 more new pages; and 

9 the instructions operable to cause a programmable processor to add the graphical 

10 elements include instructions operable to cause a programmable processor to 

1 1 add the graphical elements to the one or more new pages according to the 

12 map. 

1 6. The computer program product of claim 5, wherein the instructions operable 

2 to cause a programmable processor to add the graphical elements according to the map 

3 include instructions operable to cause a programmable processor to: 

4 associate one or more particular textual elements with one or more particular 

5 graphical elements; 

6 determine distances in the second dimension between the particular textual elements 

7 and the particular graphical elements; and 

8 select positions in the one or more new pages for placement of the particular 

9 graphical elements based on the distances and the positions of the particular textual elements 

10 in the one or more new pages as listed in the map. 

1 7. The computer program product of claim 6, wherein the instructions operable 

2 to cause a programmable processor to associate include instructions operable to cause a 

3 programmable processor to : 

4 associate a particular graphical element with a particular textual element that is 

5 nearest to the particular graphical element in the first dimension. 

1 8. The computer program product of claim 6, wherein 
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2 the instructions operable to cause a programmable processor to associate include 

3 instructions operable to cause a programmable processor to 

4 associate a given graphical element with a plurality of given textual elements; 

5 and 

6 the instructions operable to cause a programmable processor to associate include 

7 instructions operable to cause a programmable processor to 

8 scale the given graphical element when a distance in the first dimension 

9 between the plurality of given textual elements in the page differs from a corresponding 
1 o distance between the plurality of given textual elements in the one or more new pages. 

1 9. The computer program product of claim 2, wherein the textual elements are 

2 organized as words. 

1 1 0. A computer program product, tangibly stored on a computer-readable 

2 medium, for reflowing a page, comprising instructions operable to cause a programmable 

3 processor to: 

4 receive a page represented in a page description language, the page including a 

5 plurality of page objects; and 

6 change a size of the page objects without changing the size of the page, while 

7 maintaining spatial relationships between the page objects in a dimension of the page. 

1 11. The computer program product of claim 1 0, wherein the page objects include 

2 textual and graphical elements, and wherein the instructions operable to cause a 

3 programmable processor to change include instructions operable to cause a programmable 

4 processor to: 

5 maintain spatial relationships between the textual and graphical elements in the 

6 dimension. 

1 12. The computer program product of claim 1 1 , wherein the instructions operable 

2 to cause a programmable processor to change include instructions operable to cause a 

3 programmable processor to : 

4 create one or more new pages having the same size as the page in a further 

5 dimension; 
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6 scale the textual and graphical elements, producing scaled textual and graphical 

7 elements; and 

8 add the scaled textual and graphical elements to the one or more new pages. 

1 13. The computer program product of claim 12, wherein the instructions operable 

2 to cause a programmable processor to add include instructions operable to cause a 

3 programmable processor to: 

4 identify distances in the dimension between one or more textual elements and one or 

5 more graphical elements; 

6 add the scaled textual elements to the one or more new pages; and 

7 add the scaled graphical elements to the one or more new pages based on the 

8 distances in the dimension and the positions of the scaled textual elements in the one or more 

9 new pages. 

1 14. The computer program product of claim 13, wherein: 

2 the instructions operable to cause a programmable processor to identify include 

3 instructions operable to cause a programmable processor to 

4 create a map containing the positions of the textual elements in the page, and 

5 augment the map with the positions of the textual elements in the one or more 

6 new pages to produce a relationship for each textual element between the position of 

7 the textual element in the page and the position of the corresponding scaled textual 

8 element in the one or more new pages; and 

9 the instructions operable to cause a programmable processor to add the graphical 

10 elements include instructions operable to cause a programmable processor to 

1 1 add the graphical elements to the one or more new pages according to the map. 

1 15. The computer program product of claim 14, wherein the instructions operable 

2 to cause a programmable processor to add include instructions operable to cause a 

3 programmable processor to: 

4 associate one or more particular textual elements with one or more particular 

5 graphical elements; 
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6 determine distances in the dimension between the particular textual elements and the 

7 particular graphical elements; and 

8 select positions in the one or more new pages for placement of the scaled graphical 

9 elements corresponding to the particular graphical elements based on the distances and the 

10 positions of scaled textual elements corresponding to the particular textual elements in the 

1 1 one or more new pages as listed in the map. 

1 16. The computer program product of claim 15, wherein the instructions operable 

2 to cause a programmable processor to associate include instructions operable to cause a 

3 programmable processor to : 

4 associate a particular graphical element with a particular textual element that is 

5 nearest to the particular graphical element in the further dimension. 

1 17. The computer program product of claim 15, wherein 

2 the instructions operable to cause a programmable processor to associate include 

3 instructions operable to cause a programmable processor to associate a given graphical 

4 element with a plurality of given textual elements; and 

5 the instructions operable to cause a programmable processor to select include 

6 instructions operable to cause a programmable processor to scale the given graphical element 

7 in the dimension when a distance in the dimension between the plurality of given textual 

8 elements in the page differs from a corresponding distance in the one or more new pages 

9 between a plurality of scaled textual elements corresponding to the plurality of given textual 

10 elements. 

1 18. The computer program product of claim 1 1 , wherein the textual elements are 

2 organized as words. 

1 19. A computer program product, tangibly stored on a computer-readable 

2 medium, for reflowing a page, comprising instructions operable to cause a programmable 

3 processor to: 

4 receive a page represented in a page description language, the page including a 

5 plurality of page objects; and 
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6 change a size of the page to a changed size in a first dimension, and changing a size 

7 of one or more of the page objects, while maintaining spatial relationships between the page 

8 objects in a second dimension. 

1 20. The computer program product of claim 19, wherein the page objects include 

2 textual and graphical elements, and wherein the instructions operable to cause a 

3 programmable processor to change include instructions operable to cause a programmable 

4 processor to: 

5 maintain spatial relationships between the textual and graphical elements in the 

6 second dimension. 

1 21 . The computer program product of claim 20, wherein the instructions operable 

2 to cause a programmable processor to change include instructions operable to cause a 

3 programmable processor to: 

4 create one or more new pages having the same size as the page in a further 

5 dimension; 

6 scale the textual and graphical elements, producing scaled textual and graphical 

7 elements; and 

8 add the scaled textual and graphical elements to the one or more new pages. 

1 22. The computer program product of claim 2 1 , wherein the instructions operable 

2 to cause a programmable processor to add include instructions operable to cause a 

3 programmable processor to : 

4 identify distances in the second dimension between one or more textual elements and 

5 one or more graphical elements; 

6 add the scaled textual elements to the one or more new pages; and 

7 add the scaled graphical elements to the one or more new pages based on the 

8 distances in the second dimension and positions of the textual elements in the one or more 

9 new pages. 

1 23. The computer program product of claim 22, wherein: 

2 the instructions operable to cause a programmable processor to identify include 

3 instructions operable to cause a programmable processor to 
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create a map containing the positions of the textual elements in the page, and 
augment the map with the positions of the textual elements in the one or more 
new pages to produce a relationship for each textual element between the position of 
the textual element in the page and the position of the corresponding scaled textual 
element in the one or more new pages; and 

the instructions operable to cause a programmable processor to add the graphical 
elements include instructions operable to cause a programmable processor to 

add the graphical elements to the one or more new pages according to the map. 

24. The computer program product of claim 23, wherein the instructions operable 
to cause a programmable processor to add the graphical elements according to the map 
include instructions operable to cause a programmable processor to: 

associate one or more particular textual elements with one or more particular 
graphical elements; 

determine distances in the second dimension between the particular textual elements 
and the particular graphical elements; and 

select positions in the one or more new pages for placement of the scaled graphical 
elements corresponding to the particular graphical elements based on the distances and the 
positions of scaled textual elements corresponding to the particular textual elements in the 
one or more new pages as listed in the map. 

25. The computer program product of claim 24, wherein the instructions operable 
to cause a programmable processor to associate include instructions operable to cause a 
programmable processor to: 

associate a particular graphical element with a particular textual element that is 
nearest to the particular graphical element in the first dimension. 

26. The computer program product of claim 24, wherein 

the instructions operable to cause a programmable processor to associate include 
instructions operable to cause a programmable processor to associate a given graphical 
element with a plurality of given textual elements; and 
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5 the instructions operable to cause a programmable processor to select include 

6 instructions operable to cause a programmable processor to scale the given graphical element 

7 when a distance in the dimension between the plurality of given textual elements in the page 

8 differs from a corresponding distance in the one or more new pages between a plurality of 

9 scaled textual elements corresponding to the plurality of given textual elements. 

1 27. The computer program product of claim 20, wherein the textual elements are 

2 organized as words. 

1 28. The computer program product of claim 19, wherein the amount of size 

2 change of a page object depends on the type of the page object. 

1 29. A method for reflowing a page, comprising: 

2 receiving a page represented in a page description language, the page including a 

3 plurality of page objects; and 

4 changing a size of the page to a changed size in a first dimension without changing 

5 the size of the page objects, while maintaining spatial relationships between the page objects 

6 in a second dimension. 

1 30. The method of claim 29, wherein the page objects include textual and 

2 graphical elements, and wherein the changing step comprises: 

3 maintaining spatial relationships between the textual and graphical elements in the 

4 second dimension. 

1 31. The method of claim 30, wherein the changing step further comprises: 

2 creating one or more new pages having the changed size in the first dimension; and 

3 adding the textual and graphical elements to the one or more new pages. 

1 32. The method of claim 3 1 , wherein the adding step comprises: 

2 identifying distances in the second dimension between one or more textual elements 

3 and one or more graphical elements; 

4 adding the textual elements to the one or more new pages; and 

5 adding the graphical elements to the one or more new pages based on the distances in 

6 the second dimension and positions of the textual elements in the one or more new pages. 
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1 33. The method of claim 32, wherein: 

2 the identifying step includes 

3 creating a map containing the positions of the textual elements in the page, 

4 and 

5 augmenting the map with the positions of the textual elements in the one or 

6 more new pages to produce a relationship for each textual element between the 

7 position of the textual element in the page and the position of the textual element in 

8 the one or more new pages; and 

9 the step of adding the graphical elements includes 

10 adding the graphical elements to the one or more new pages according to the map. 

1 34. The method of claim 33, wherein the step of adding the graphical elements 

2 according to the map comprises: 

3 associating one or more particular textual elements with one or more particular 

4 graphical elements; 

5 determining distances in the second dimension between the particular textual 

6 elements and the particular graphical elements; and 

7 selecting positions in the one or more new pages for placement of the particular 

8 graphical elements based on the distances and the positions of the particular textual elements 

9 in the one or more new pages as listed in the map. 

1 35. The method of claim 34, wherein the associating step comprises: 

2 associating a particular graphical element with a particular textual element that is 

3 nearest to the particular graphical element in the first dimension. 

1 36. The method of claim 34, wherein 

2 the associating step includes associating a given graphical element with a plurality of 

3 given textual elements; and 

4 the selecting step includes scaling the given graphical element when a distance in the 

5 first dimension between the plurality of given textual elements in the page differs from a 

6 corresponding distance between the plurality of given textual elements in the one or more 

7 new pages. 
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1 37. The method of claim 30, wherein the textual elements are organized as words. 

1 38. A method for reflowing a page, comprising: 

2 receiving a page represented in a page description language, the page including a 

3 plurality of page objects; and 

4 changing a size of the page objects without changing the size of the page, while 

5 maintaining spatial relationships between the page objects in a dimension of the page. 

1 39. The method of claim 38, wherein the page objects include textual and 

2 graphical elements, and wherein the changing step comprises: 

3 maintaining spatial relationships between the textual and graphical elements in the 

4 dimension. 

1 40. The method of claim 39, wherein the changing step further comprises: 

2 creating one or more new pages having the same size as the page in a further 

3 dimension; 

4 scaling the textual and graphical elements, producing scaled textual and graphical 

5 elements; and 

6 adding the scaled textual and graphical elements to the one or more new pages. 

1 41 . The method of claim 40, wherein the adding step comprises: 

2 identifying distances in the dimension between one or more textual elements and one 

3 or more graphical elements; 

4 adding the scaled textual elements to the one or more new pages; and 

5 adding the scaled graphical elements to the one or more new pages based on the 

6 distances in the dimension and the positions of the scaled textual elements in the one or more 

7 new pages. 

1 42. The method of claim 4 1 , wherein: 

2 the identifying step includes 

3 creating a map containing the positions of the textual elements in the page, 

4 and 
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5 augmenting the map with the positions of the textual elements in the one or 

6 more new pages to produce a relationship for each textual element between the 

7 position of the textual element in the page and the position of the corresponding 

8 scaled textual element in the one or more new pages; and 

9 the step of adding the graphical elements includes 

10 adding the graphical elements to the one or more new pages according to the map. 

1 43. The method of claim 42, wherein the step of adding the scaled graphical 

2 elements according to the map comprises: 

3 associating one or more particular textual elements with one or more particular 

4 graphical elements; 

5 determining distances in the dimension between the particular textual elements and 

6 the particular graphical elements; and 

7 selecting positions in the one or more new pages for placement of the scaled graphical 

8 elements corresponding to the particular graphical elements based on the distances and the 

9 positions of scaled textual elements corresponding to the particular textual elements in the 

1 0 one or more new pages as listed in the map. 

1 44. The method of claim 43, wherein the associating step comprises: 

2 associating a particular graphical element with a particular textual element that is 

3 nearest to the particular graphical element in the further dimension. 

1 45. The method of claim 43, wherein 

2 the associating step includes associating a given graphical element with a plurality of 

3 given textual elements; and 

4 the selecting step includes scaling the given graphical element in the dimension when 

5 a distance in the dimension between the plurality of given textual elements in the page differs 

6 from a corresponding distance in the one or more new pages between a plurality of scaled 

7 textual elements corresponding to the plurality of given textual elements. 

1 46. The method of claim 39, wherein the textual elements are organized as words. 

1 47. A method for reflowing a page, comprising: 
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2 receiving a page represented in a page description language, the page including a 

3 plurality of page objects; and 

4 changing a size of the page to a changed size in a first dimension, and changing a size 

5 of one or more of the page objects, while maintaining spatial relationships between the page 

6 objects in a second dimension. 

1 48. The method of claim 47, wherein the page objects include textual and 

2 graphical elements, and wherein the changing step comprises: 

3 maintaining spatial relationships between the textual and graphical elements in the 

4 second dimension. 

1 49. The method of claim 48, wherein the changing step further comprises: 

2 creating one or more new pages having the same size as the page in a further 

3 dimension; 

4 scaling the textual and graphical elements, producing scaled textual and graphical 

5 elements; and 

6 adding the scaled textual and graphical elements to the one or more new pages. 

1 50. The method of claim 49, wherein the adding step comprises: 

2 identifying distances in the second dimension between one or more textual elements 

3 and one or more graphical elements; 

4 adding the scaled textual elements to the one or more new pages; and 

5 adding the scaled graphical elements to the one or more new pages based on the 

6 distances in the second dimension and positions of the textual elements in the one or more 

7 new pages. 

1 51. The method of claim 50, wherein: 

2 the identifying step includes 

3 creating a map containing the positions of the textual elements in the page, 

4 and 

5 augmenting the map with the positions of the textual elements in the one or 

6 more new pages to produce a relationship for each textual element between the 
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7 position of the textual element in the page and the position of the corresponding 

8 scaled textual element in the one or more new pages; and 

9 the step of adding the graphical elements includes 

10 adding the graphical elements to the one or more new pages according to the map. 

1 52. The method of claim 5 1 , wherein the step of adding the graphical elements 

2 according to the map comprises: 

3 associating one or more particular textual elements with one or more particular 

4 graphical elements; 

5 determining distances in the second dimension between the particular textual 

6 elements and the particular graphical elements; and 

7 selecting positions in the one or more new pages for placement of the scaled graphical 

8 elements corresponding to the particular graphical elements based on the distances and the 

9 positions of scaled textual elements corresponding to the particular textual elements in the 

1 0 one or more new pages as listed in the map. 

1 53. The method of claim 52, wherein the associating step comprises: 

2 associating a particular graphical element with a particular textual element that is 

3 nearest to the particular graphical element in the first dimension. 

1 54. The method of claim 52, wherein 

2 the associating step includes associating a given graphical element with a plurality of 

3 given textual elements; and 

4 the selecting step includes scaling the given graphical element when a distance in the 

5 dimension between the plurality of given textual elements in the page differs from a 

6 corresponding distance in the one or more new pages between a plurality of scaled textual 

7 elements corresponding to the plurality of given textual elements. 

1 55. The method of claim 48, wherein the textual elements are organized as words. 

1 56. The method of claim 47, wherein the amount of size change of a page object 

2 depends on the type of the page object. 
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1 57. A computer program product, tangibly stored on a computer-readable 

2 medium, for forming illustrations in a page, comprising instructions operable to cause a 

3 programmable processor to : 

4 receive a page represented in a page description language (PDL), the page including a 

5 plurality of page objects including line art elements, each page object associated with a PDL 

6 element range including at least one PDL element; 

7 repeatedly augment each PDL range with a PDL element that is adjacent to the PDL 

8 range and is not part of another PDL range when the bounding box of the PDL element 

9 overlaps the bounding box of the PDL object associated with the PDL range; 
10 such that the PDL elements in each PDL range define an illustration. 

1 58. The computer program product of claim 57, further comprising instructions 

2 operable to cause a programmable processor to: 

3 combine two illustrations when their PDL ranges are adjacent and their bounding 

4 boxes overlap. 

1 59. A computer program product, tangibly stored on a computer-readable 

2 medium, for forming illustrations in a page, comprising instructions operable to cause a 

3 programmable processor to : 

4 receive a page represented in a page description language (PDL) ? the page including a 

5 plurality of page objects including line art elements, each page object associated with a PDL 

6 element range including at least one PDL element; 

7 recursively coalesce line art elements having overlapping bounding boxes to form one 

8 or more illustrations; 

9 add to each illustration each PDL element within the PDL range of the illustration 

10 that is not part of the illustration when the bounding box of the PDL element overlaps the 

1 1 bounding box of the illustration; and 

1 2 repeatedly augment the PDL range of each illustration with PDL elements that are 

13 adjacent to the PDL range and are not part of another illustration when the bounding box of 

14 the PDL elements overlap the bounding box of the illustration; 

15 such that the PDL elements in each PDL range define an illustration. 
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1 60. The computer program product of claim 59, further comprising instructions 

2 operable to cause a programmable processor to: 

3 combine two illustrations when their PDL ranges are adjacent and their bounding 

4 boxes overlap. 

1 61 . The computer program product of claim 60, wherein the instructions operable 

2 to cause a programmable processor to recursively coalesce comprises instructions operable to 

3 cause a programmable processor to: 

4 combine two line art elements having overlapping bounding boxes, thereby forming 

5 an illustration; and 

6 create a new bounding box containing the illustration. 

1 62. The computer program product of claim 61 , further comprising instructions 

2 operable to cause a programmable processor to: 

3 combine a line art element with the illustration when the bounding boxes of the line 

4 art element and the illustration overlap. 

1 63 . A method for forming illustrations in a page, comprising: 

2 receiving a page represented in a page description language (PDL), the page 

3 including a plurality of page objects including line art elements, each page object associated 

4 with a PDL element range including at least one PDL element; 

5 repeatedly augmenting each PDL range with a PDL element that is adjacent to the 

6 PDL range and is not part of another PDL range when the bounding box of the PDL element 

7 overlaps the bounding box of the PDL object associated with the PDL range; 

8 such that the PDL elements in each PDL range define an illustration. 

1 64. The method of claim 63, further comprising: 

2 combining two illustrations when their PDL ranges are adjacent and their bounding 

3 boxes overlap. 

1 65. A method for forming illustrations in a page, comprising: 
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2 receiving a page represented in a page description language (PDL), the page 

3 including a plurality of page objects including line art elements, each page object associated 

4 with a PDL element range including at least one PDL element; 

5 recursively coalescing line art elements having overlapping bounding boxes to form 

6 one or more illustrations; 

7 adding to each illustration each PDL element within the PDL range of the illustration 

8 that is not part of the illustration when the bounding box of the PDL element overlaps the 

9 bounding box of the illustration; and 

10 repeatedly augmenting the PDL range of each illustration with PDL elements that are 

1 1 adjacent to the PDL range and are not part of another illustration when the bounding box of 

12 the PDL elements overlap the bounding box of the illustration; 

1 3 such that the PDL elements in each PDL range define an illustration. 

1 66. The method of claim 65, further comprising: 

2 combining two illustrations when their PDL ranges are adjacent and their bounding 

3 boxes overlap. 

1 67. The method of claim 66, wherein the step of recursively coalescing comprises: 

2 combining two line art elements having overlapping bounding boxes, thereby forming 

3 an illustration; and 

4 creating a new bounding box containing the illustration. 

1 68. The method of claim 67, further comprising: 

2 combining a line art element with the illustration when the bounding boxes of the line 

3 art element and the illustration overlap. 
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ABSTRACT OF THE DISCLOSURE 

A method and computer program product for reflowing a PDL page without using the 
original application used to create the PDL page. The method and computer program 
product include receiving a page represented in a page description language, the page 
including a plurality of page objects, and changing one or both of a size of the page and a 
size of one or more of the page objects, while maintaining spatial relationships between the 
page objects. 
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COMBINED DECLARATION AND POWER OF ATTORNEY 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name, 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and 
joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is 
sought on the invention entitled TEXT REFLOW IN A STRUCTURED DOCUMENT, the specification of 
which: 

[X] is attached hereto. 

[] was filed on _ as Application Serial No. _ and was amended on . 

[] was described and claimed in PCT International Application No. filed on 

and as amended under PCT Article 19 on . 

I hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose all information I know to be material to patentability in accordance with 
Title 37, Code of Federal Regulations, §1.56. 

I hereby claim the benefit under Title 35, United States Code, §1 19(e)(1) of any United States provisional 
application(s) listed below: 

U.S. Serial No. Filing Date Status 



I hereby claim the benefit under Title 35, United States Code, § 120 of any United States application(s) 
listed below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States application in the manner provided by the first paragraph of Title 35, United States Code, §1 12, 1 
acknowledge the duty to disclose all information I know to be material to patentability as defined in Title 37, Code 
of Federal Regulations, § 1 .56(a) which became available between the filing date of the prior application and the 
national or PCT international filing date of this application: 

U.S. Serial No. Filing Date Status 



I hereby appoint the following attorneys and/or agents to prosecute this application and to transact all 
business in the Patent and Trademark Office connected therewith, and direct that all correspondence be addressed to 
that Customer Number: 

Customer Number 021876 

Address all telephone calls to RICHARD A. DUNNING, JR., Reg. No. 42,502 at telephone number (650) 
322-5070. 

[X] For Assigned Inventions: I understand that the purpose of making this appointment is to permit 
prosecution of patent applications for the above-identified invention for the benefit of my assignee, and that this 
appointment does not create an attorney-client relationship between me and these appointees. 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the knowledge 
that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 



1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the validity of the 
application or any patents issued thereon. 
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Date: Z^L \l/V\A Q~Q 
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Residence Address: Menlo Park, California 94025 
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Post Office Address: 145 Campo Bello Lane, Menlo Park, CA 94025 
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